1
00:00:20,400 --> 00:00:22,720
SIREN WAILS

2
00:00:24,120 --> 00:00:27,000
A car crashes
in the middle of the night.

3
00:00:31,560 --> 00:00:33,360
A woman is run down.

4
00:00:34,720 --> 00:00:36,840
SOBBING

5
00:00:38,760 --> 00:00:41,800
It's the first death of its kind
in the world...

6
00:00:45,400 --> 00:00:48,480
..a pedestrian killed by
a driverless car.

7
00:00:54,800 --> 00:00:57,240
Although the vehicle
<font color="

8
00:00:57,240 --> 00:00:59,480
using artificial intelligence,

9
00:00:59,480 --> 00:01:02,000
there was a human operator
behind the wheel.

10
00:01:08,880 --> 00:01:12,920
For the first time ever,
the decision had to be made -

11
00:01:12,920 --> 00:01:15,680
who was to blame for this crash?

12
00:01:15,680 --> 00:01:18,480
Was it the human or the AI?

13
00:01:21,200 --> 00:01:25,640
<font color="
a machine beyond the mind of man.

14
00:01:25,640 --> 00:01:30,760
For decades, scientists have dreamed
of creating incredible machines

15
00:01:30,760 --> 00:01:35,920
that could talk like us,
learn like us, think like us.

16
00:01:38,080 --> 00:01:43,200
But what we didn't imagine
is the impact they would have on us.

17
00:01:43,200 --> 00:01:46,000
<font color="
I'm exploring what happens

18
00:01:46,000 --> 00:01:49,120
when AI collides with human lives,

19
00:01:49,120 --> 00:01:54,240
unearthing stories far stranger than
we could ever have imagined.

20
00:02:19,800 --> 00:02:22,480
I'm Professor Hannah Fry.
I'm a mathematician,

21
00:02:22,480 --> 00:02:24,680
and I've spent my career
<font color="

22
00:02:24,680 --> 00:02:27,320
technology can transform our future.

23
00:02:29,040 --> 00:02:32,560
Here in Phoenix,
a multi-billion-dollar industry

24
00:02:32,560 --> 00:02:35,880
has the potential
to save millions of lives.

25
00:02:35,880 --> 00:02:37,720
That is weird.

26
00:02:37,720 --> 00:02:39,520
SHE CHUCKLES

27
00:02:39,520 --> 00:02:42,760
We're on the cusp of
<font color="

28
00:02:42,760 --> 00:02:45,440
and driverless taxis
are now a reality

29
00:02:45,440 --> 00:02:47,800
in several countries
around the world.

30
00:02:49,120 --> 00:02:50,760
All right, request it.

31
00:02:50,760 --> 00:02:53,280
You hail them on an app.

32
00:02:53,280 --> 00:02:56,200
Ooh! Love my initials on top.

33
00:02:56,200 --> 00:02:57,720
This is my first time
<font color="

34
00:02:57,720 --> 00:02:59,920
Worked with Google for years.

35
00:02:59,920 --> 00:03:02,680
Never been in a driverless car.

36
00:03:02,680 --> 00:03:04,080
Imagine.

37
00:03:04,080 --> 00:03:05,600
Here it is!

38
00:03:06,640 --> 00:03:08,280
Where's it's going to stop?

39
00:03:10,320 --> 00:03:12,840
Where's it going to stop?

40
00:03:12,840 --> 00:03:14,200
There?

41
00:03:14,200 --> 00:03:16,360
Ooh!

42
00:03:16,360 --> 00:03:18,760
Oh, that was quite well done.

43
00:03:18,760 --> 00:03:20,480
<font color="

44
00:03:24,320 --> 00:03:26,200
No-one in the car.

45
00:03:26,200 --> 00:03:28,160
Start the ride.

46
00:03:28,160 --> 00:03:30,040
Go!

47
00:03:30,040 --> 00:03:33,120
SHE GASPS AND LAUGHS

48
00:03:33,120 --> 00:03:35,280
It's quite nerve-racking!

49
00:03:35,280 --> 00:03:36,880
AUTOMATED VOICE: We'll do
all the driving,

50
00:03:36,880 --> 00:03:41,600
so please don't touch the steering
<font color="

51
00:03:41,600 --> 00:03:43,320
It's green, but it slowed down.

52
00:03:43,320 --> 00:03:45,160
Oh, it's turning right.
That's why.

53
00:03:48,320 --> 00:03:50,400
I sort of don't trust it.
I don't trust it.

54
00:03:51,440 --> 00:03:53,680
Ooh, we're changing lanes.

55
00:03:53,680 --> 00:03:56,000
SHE LAUGHS

56
00:03:58,800 --> 00:04:00,080
It's not timid, is it?

57
00:04:01,960 --> 00:04:04,840
<font color="
to drive itself

58
00:04:04,840 --> 00:04:07,760
is nothing short of miraculous.

59
00:04:07,760 --> 00:04:10,440
The car uses a combination
of sensors

60
00:04:10,440 --> 00:04:13,280
to understand what's going on
around it.

61
00:04:15,000 --> 00:04:18,920
First, cameras -
great at identifying road signs,

62
00:04:18,920 --> 00:04:22,440
<font color="
and other cars,

63
00:04:22,440 --> 00:04:25,120
but they struggle in bad weather.

64
00:04:26,880 --> 00:04:30,400
So, many driverless cars
also have radar,

65
00:04:30,400 --> 00:04:32,760
often built into the front bumper,

66
00:04:32,760 --> 00:04:36,160
which sends out radio waves
that bounce off objects

67
00:04:36,160 --> 00:04:39,080
<font color="

68
00:04:39,080 --> 00:04:43,640
It works well over long distances,
but not so well up close.

69
00:04:44,800 --> 00:04:48,440
That's why some driverless cars
also use lidar,

70
00:04:48,440 --> 00:04:52,880
the spinning cylinders you sometimes
see on the roof and side of the car.

71
00:04:52,880 --> 00:04:55,160
It's like radar,
<font color="

72
00:04:55,160 --> 00:05:00,520
and is able to build up a detailed
3D picture of nearby objects.

73
00:05:02,800 --> 00:05:05,080
Precise but easily confused

74
00:05:05,080 --> 00:05:09,360
by reflective surfaces
like windows and shiny buildings.

75
00:05:10,920 --> 00:05:15,640
This is sort of inside
the computer's brain, as it were.

76
00:05:15,640 --> 00:05:19,200
<font color="
what it thinks is around you.

77
00:05:19,200 --> 00:05:22,680
The AI takes these
three imperfect systems,

78
00:05:22,680 --> 00:05:25,120
pieces together what's actually
out there,

79
00:05:25,120 --> 00:05:27,360
tries to predict
what will happen next

80
00:05:27,360 --> 00:05:31,680
and decides how the car should
<font color="

81
00:05:32,840 --> 00:05:36,360
When I was writing about
these things ten years ago,

82
00:05:36,360 --> 00:05:38,000
they were a research project.

83
00:05:38,000 --> 00:05:39,560
They were understanding
the environment,

84
00:05:39,560 --> 00:05:43,400
advancing the engineering,
kind of tweaking the software.

85
00:05:43,400 --> 00:05:45,360
<font color="
that they've become

86
00:05:45,360 --> 00:05:49,680
commercially available,
that just anybody could hail one.

87
00:05:53,320 --> 00:05:56,240
And soon they're even coming
to the UK,

88
00:05:56,240 --> 00:05:59,120
starting with the busy streets
of London.

89
00:06:01,280 --> 00:06:04,320
It is good at sticking to
<font color="

90
00:06:07,880 --> 00:06:11,160
I think there is actually
quite a lot to look forward to

91
00:06:11,160 --> 00:06:14,040
from this future
with driverless cars.

92
00:06:14,040 --> 00:06:16,080
Humans make terrible drivers.

93
00:06:16,080 --> 00:06:17,760
And these things, they're not...

94
00:06:17,760 --> 00:06:19,320
They're not falling asleep
<font color="

95
00:06:19,320 --> 00:06:22,680
They're not drink-driving.
They're not getting road rage.

96
00:06:22,680 --> 00:06:25,480
Like, fine. Maybe they're not
going to be perfect,

97
00:06:25,480 --> 00:06:27,600
but there is a lot of scope

98
00:06:27,600 --> 00:06:30,840
for roads to be much safer than
they currently are.

99
00:06:30,840 --> 00:06:34,360
<font color="
that is worth pursuing.

100
00:06:35,840 --> 00:06:38,240
But the story of how we got
to this point

101
00:06:38,240 --> 00:06:41,480
is marked by tragedy and loss.

102
00:06:44,360 --> 00:06:47,240
There was a pedestrian
walking a bicycle.

103
00:06:47,240 --> 00:06:49,240
Once the pedestrian got into
the lane of traffic,

104
00:06:49,240 --> 00:06:51,480
<font color="

105
00:06:51,480 --> 00:06:53,400
It was a self-driving vehicle.

106
00:06:54,960 --> 00:06:57,360
It was in the autonomous mode
at the time.

107
00:06:57,360 --> 00:07:02,080
In Arizona, in 2018,
for the first time,

108
00:07:02,080 --> 00:07:04,760
someone was struck
and fatally injured

109
00:07:04,760 --> 00:07:06,840
by a driverless car.

110
00:07:06,840 --> 00:07:09,600
<font color="
was the human backup driver

111
00:07:09,600 --> 00:07:13,520
in the self-driving Uber vehicle
that hit and killed Elaine Herzberg.

112
00:07:15,600 --> 00:07:18,080
After years of
avoiding the limelight,

113
00:07:18,080 --> 00:07:20,120
the woman at the centre
of the story,

114
00:07:20,120 --> 00:07:23,080
<font color="
had agreed to meet me.

115
00:07:24,680 --> 00:07:26,400
Hey! Hi, how are you?
Good. How are you doing?

116
00:07:26,400 --> 00:07:28,520
Do you want to come in?
Yeah, thank you.

117
00:07:28,520 --> 00:07:31,400
You haven't spoken on camera
about this before, have you? No.

118
00:07:31,400 --> 00:07:32,960
<font color="
I don't know.

119
00:07:32,960 --> 00:07:35,680
I have a whole plethora of emotions
going through me,

120
00:07:35,680 --> 00:07:38,520
but my biggest issue
going through this

121
00:07:38,520 --> 00:07:42,560
was not being able to rebut anything
or even defend myself.

122
00:07:42,560 --> 00:07:44,520
It's like getting a chance
<font color="

123
00:07:44,520 --> 00:07:48,840
Yeah, cos I've been dealing with it
now for, what, seven years?

124
00:07:48,840 --> 00:07:50,640
So this is you? Mm-hm. Yes.

125
00:07:50,640 --> 00:07:53,440
And that's to get you
in and out of the buildings.

126
00:07:53,440 --> 00:07:57,840
Rafaela got a job at Uber in 2017,
not long after the company

127
00:07:57,840 --> 00:08:02,080
<font color="
their own self-driving cars.

128
00:08:02,080 --> 00:08:04,600
I like a lot of technology stuff.

129
00:08:04,600 --> 00:08:08,080
Self-driving vehicles
were starting to emerge.

130
00:08:08,080 --> 00:08:12,040
I knew that Phoenix was becoming
a hotbed for it,

131
00:08:12,040 --> 00:08:15,520
so I would see autonomous vehicles
<font color="

132
00:08:15,520 --> 00:08:19,200
I'm interested in that,
so I went and applied.

133
00:08:19,200 --> 00:08:20,880
I passed everything instantly.

134
00:08:20,880 --> 00:08:22,640
So, I was excited. Mmm.

135
00:08:23,760 --> 00:08:26,360
Her job was to ride
in the autonomous cars

136
00:08:26,360 --> 00:08:29,160
as Uber began testing
<font color="

137
00:08:29,160 --> 00:08:31,560
on Arizona's public roads.

138
00:08:31,560 --> 00:08:33,680
There was a lot of
engineers, coders,

139
00:08:33,680 --> 00:08:35,440
but everybody was super nice.

140
00:08:35,440 --> 00:08:38,040
And you have this chance to see
the new technology from the inside.

141
00:08:38,040 --> 00:08:40,760
Not just see it, I'm testing it
<font color="

142
00:08:40,760 --> 00:08:44,000
I loved my job.
I absolutely loved my job.

143
00:08:44,000 --> 00:08:47,160
When you were in the cars
in those early days,

144
00:08:47,160 --> 00:08:49,680
how good did you think they were
at driving?

145
00:08:49,680 --> 00:08:52,200
They were better than what I thought
<font color="

146
00:08:52,200 --> 00:08:55,240
And it also depends. Like,
Uber vehicles for the longest time

147
00:08:55,240 --> 00:08:59,680
had problems with overreacting
with things on the side of the road.

148
00:08:59,680 --> 00:09:00,920
And some builds are great,

149
00:09:00,920 --> 00:09:03,520
but then they do another update
to try to fix something else

150
00:09:03,520 --> 00:09:05,440
<font color="
go haywire.

151
00:09:05,440 --> 00:09:07,920
And then you just don't know.

152
00:09:07,920 --> 00:09:10,960
So much has happened to you
since that, right?

153
00:09:10,960 --> 00:09:12,000
Yeah.

154
00:09:19,200 --> 00:09:21,920
During its first year of testing
in Phoenix,

155
00:09:21,920 --> 00:09:25,720
Uber assigned two operators
<font color="

156
00:09:25,720 --> 00:09:28,640
All right, and we're engaging.

157
00:09:28,640 --> 00:09:31,760
One rode in the passenger seat
to track the car's performance.

158
00:09:31,760 --> 00:09:35,520
On the laptop, I can monitor
a lot of the prediction software.

159
00:09:35,520 --> 00:09:37,760
So I can see, like,
where the car is going.

160
00:09:37,760 --> 00:09:41,240
<font color="
had to be logged on a computer.

161
00:09:42,800 --> 00:09:45,200
The second operator
sat behind the wheel

162
00:09:45,200 --> 00:09:47,760
and kept their eyes on the road.

163
00:09:47,760 --> 00:09:52,680
They were expected to take control
if the AI in the car malfunctioned.

164
00:10:01,360 --> 00:10:05,560
But a year into testing,
<font color="

165
00:10:05,560 --> 00:10:08,240
Now one operator
had to watch the road

166
00:10:08,240 --> 00:10:10,760
AND monitor the car's actions.

167
00:10:12,280 --> 00:10:14,240
OK. Come on, then.

168
00:10:14,240 --> 00:10:17,920
This decision to switch
to a single human in the car

169
00:10:17,920 --> 00:10:22,160
came just a few months before
Rafaela's crash.

170
00:10:22,160 --> 00:10:25,600
<font color="
I still get nervous. Oh, really?

171
00:10:25,600 --> 00:10:27,160
Yeah. That's why I'm hesitating,

172
00:10:27,160 --> 00:10:30,840
because I'm just trying to not have
a panic attack. I understand.

173
00:10:30,840 --> 00:10:32,280
OK.

174
00:10:34,120 --> 00:10:37,200
Oh, sorry.
No, don't apologise, please.

175
00:10:37,200 --> 00:10:41,680
<font color="
No, I want to do this.

176
00:10:41,680 --> 00:10:44,880
Are you sure? Yeah.
OK, here we go.

177
00:10:46,760 --> 00:10:49,520
I don't want my driving
to unnerve you.

178
00:10:49,520 --> 00:10:51,920
No, I'm just worried
you're going to get pulled over

179
00:10:51,920 --> 00:10:55,160
<font color="
a typical person.

180
00:10:55,160 --> 00:10:57,120
I'm driving like a granny?

181
00:10:57,120 --> 00:10:59,280
Well, you're driving like
an autonomous vehicle, actually.

182
00:10:59,280 --> 00:11:00,720
Oh, really?
Right at the speed limit.

183
00:11:00,720 --> 00:11:02,760
THEY LAUGH

184
00:11:02,760 --> 00:11:04,960
Each operator was allocated a route

185
00:11:04,960 --> 00:11:08,120
<font color="
again and again.

186
00:11:08,120 --> 00:11:10,280
It could be monotonous work.

187
00:11:10,280 --> 00:11:12,840
Like, the most boring one I hated
was this one

188
00:11:12,840 --> 00:11:16,520
through this little neighbourhood.
You come out, you go here.

189
00:11:16,520 --> 00:11:20,120
Then this is all like 20 miles
<font color="

190
00:11:20,120 --> 00:11:22,160
and then, boom, back.

191
00:11:22,160 --> 00:11:24,720
And that's all it is.
Round and round...

192
00:11:24,720 --> 00:11:27,520
For eight hours. ..with the car
driving itself. Mm-hm.

193
00:11:29,320 --> 00:11:33,440
Humans are not good at paying
<font color="

194
00:11:33,440 --> 00:11:37,160
And with only one operator
monitoring a repetitive route,

195
00:11:37,160 --> 00:11:39,960
there was a danger of
getting distracted.

196
00:11:41,320 --> 00:11:44,040
But when it wasn't quiet
on the streets,

197
00:11:44,040 --> 00:11:47,000
there could be a lot
for one person to do.

198
00:11:49,040 --> 00:11:50,600
<font color="
isn't there?

199
00:11:50,600 --> 00:11:53,320
Yes, and this is what we're testing.
Yeah.

200
00:11:53,320 --> 00:11:55,640
And you can't predict
what somebody's going to do.

201
00:11:55,640 --> 00:11:59,640
So I would always take it out of
autonomous mode during these areas.

202
00:11:59,640 --> 00:12:02,560
Right.
<font color="

203
00:12:02,560 --> 00:12:04,760
But then all of the sudden,
they changed it.

204
00:12:06,320 --> 00:12:07,920
But there are several screens

205
00:12:07,920 --> 00:12:09,960
that you're continually
having to look at.

206
00:12:09,960 --> 00:12:13,200
Yes. We were supposed
to push buttons and enter codes

207
00:12:13,200 --> 00:12:14,760
<font color="

208
00:12:14,760 --> 00:12:16,680
So, you're trying to put something
into the iPad

209
00:12:16,680 --> 00:12:19,280
while you're supposed to be
monitoring the road.

210
00:12:19,280 --> 00:12:20,320
Yeah.

211
00:12:26,360 --> 00:12:32,720
On the night of March 18th 2018,
Rafaela was on her usual test route.

212
00:12:35,400 --> 00:12:37,200
<font color="

213
00:12:37,200 --> 00:12:41,360
and had been running for 19 minutes
without incident.

214
00:12:48,520 --> 00:12:53,280
At 9.58pm,
it turned right onto Mill Avenue.

215
00:12:58,320 --> 00:13:03,000
At the same time, a pedestrian began
walking across Mill Avenue,

216
00:13:03,000 --> 00:13:05,240
pushing a bicycle by her side.

217
00:13:15,160 --> 00:13:19,520
<font color="
as the vehicle approached the woman.

218
00:13:20,800 --> 00:13:25,520
The car should have detected her,
but it didn't.

219
00:13:33,440 --> 00:13:36,920
By the time Rafaela looked up
and slammed on the brakes,

220
00:13:36,920 --> 00:13:38,520
it was too late.

221
00:13:38,520 --> 00:13:42,760
The vehicle struck the woman
at 39mph.

222
00:13:44,120 --> 00:13:48,000
<font color="
was Elaine Herzberg.

223
00:13:48,000 --> 00:13:50,720
She was often seen on her bike
in the area.

224
00:13:52,920 --> 00:13:56,440
She never gave up.
She always helped people.

225
00:13:56,440 --> 00:13:58,520
She was funny, funny, funny, funny.

226
00:13:58,520 --> 00:14:01,640
If you were her friend,
you felt true love from her.

227
00:14:01,640 --> 00:14:03,720
<font color="
what happened to her.

228
00:14:13,160 --> 00:14:14,960
So this is the park...
That's the park. Right.

229
00:14:14,960 --> 00:14:18,840
I did... That venue, that theatre,
everybody crossed there,

230
00:14:18,840 --> 00:14:20,160
including the homeless people,

231
00:14:20,160 --> 00:14:22,560
cos the homeless people
<font color="

232
00:14:22,560 --> 00:14:23,920
That's where she was going.

233
00:14:27,560 --> 00:14:31,720
This was the first time Rafaela had
returned to the site of the crash.

234
00:14:37,200 --> 00:14:40,120
Do you see that sign on that post,
that light? Yeah, yeah.

235
00:14:40,120 --> 00:14:41,960
She was over there. Oh, gosh.

236
00:14:44,040 --> 00:14:47,680
<font color="
the whole time. Yeah.

237
00:14:47,680 --> 00:14:50,280
RAFAELA EXHALES

238
00:14:50,280 --> 00:14:51,320
I...

239
00:14:54,640 --> 00:14:56,040
The screaming, it...

240
00:14:59,280 --> 00:15:02,040
It was just terrible to hear it.
And then...

241
00:15:02,040 --> 00:15:05,440
But then what was worse
was when it stopped.

242
00:15:05,440 --> 00:15:10,600
<font color="
then they said she'd passed away,

243
00:15:10,600 --> 00:15:12,360
and then I lost it.

244
00:15:14,520 --> 00:15:15,600
Somebody died.

245
00:15:24,280 --> 00:15:28,080
Following the collision, the police
launched a criminal investigation

246
00:15:28,080 --> 00:15:31,680
into the artificial intelligence car

247
00:15:31,680 --> 00:15:33,760
and its human operator.

248
00:15:43,320 --> 00:15:46,680
<font color="
drive a car involves much more

249
00:15:46,680 --> 00:15:49,480
than turning a wheel
and pressing pedals.

250
00:15:49,480 --> 00:15:54,400
You also need to teach it to do
things that humans do instinctively,

251
00:15:54,400 --> 00:15:58,680
like spotting pedestrians
and recognising road signs.

252
00:15:58,680 --> 00:16:00,760
<font color="

253
00:16:00,760 --> 00:16:05,120
but training a machine to do it
has proved incredibly difficult.

254
00:16:06,120 --> 00:16:10,040
So back in the early days of AI,
the only real form of intelligence

255
00:16:10,040 --> 00:16:12,320
that we knew about
was human intelligence.

256
00:16:12,320 --> 00:16:17,000
And so people looked to
<font color="

257
00:16:17,000 --> 00:16:21,680
about how to build
an electronic brain.

258
00:16:21,680 --> 00:16:24,080
And the thing about the human brain
is that it is made up

259
00:16:24,080 --> 00:16:26,880
by these billions and billions
of neurones

260
00:16:26,880 --> 00:16:29,400
that are connected together.

261
00:16:29,400 --> 00:16:30,560
And as you think,

262
00:16:30,560 --> 00:16:33,440
<font color="
these little electrical impulses,

263
00:16:33,440 --> 00:16:36,720
these little bursts
that can be big or small,

264
00:16:36,720 --> 00:16:38,800
through this network in your brain.

265
00:16:40,720 --> 00:16:43,680
In the 1950s, people started
trying to construct

266
00:16:43,680 --> 00:16:46,080
a much simpler computer version

267
00:16:46,080 --> 00:16:48,440
<font color="
artificial neurones

268
00:16:48,440 --> 00:16:51,080
would pass signals
between each other.

269
00:16:51,080 --> 00:16:55,320
This concept became what we now call
a neural network.

270
00:16:57,360 --> 00:17:01,920
But rather than all of your neurones
being kind of intermeshed together,

271
00:17:01,920 --> 00:17:05,200
they appear in layers,
<font color="

272
00:17:08,560 --> 00:17:12,360
Decades later, people realised
you could use these neural networks

273
00:17:12,360 --> 00:17:15,480
to recognise images -
like a stop sign.

274
00:17:16,760 --> 00:17:19,560
Here's a simplified version
of how it works.

275
00:17:21,080 --> 00:17:24,080
Each neurone in the hierarchy
has its own job.

276
00:17:24,080 --> 00:17:27,760
<font color="
at a single pixel each.

277
00:17:27,760 --> 00:17:29,920
And as you go further up
the network,

278
00:17:29,920 --> 00:17:31,920
things get more sophisticated.

279
00:17:32,920 --> 00:17:34,880
Maybe there'll be a neurone up there

280
00:17:34,880 --> 00:17:38,200
that is checking to see
if there's some red.

281
00:17:38,200 --> 00:17:40,040
<font color="
that's checking to see

282
00:17:40,040 --> 00:17:42,600
if there's an octagonal shape

283
00:17:42,600 --> 00:17:45,440
that stands out from the background
behind it.

284
00:17:45,440 --> 00:17:48,400
And all of this information,
all of these signals get sent up,

285
00:17:48,400 --> 00:17:52,240
so you eventually get right to
<font color="

286
00:17:52,240 --> 00:17:54,200
to the big boss,

287
00:17:54,200 --> 00:17:57,040
who makes a decision based on
all of the information

288
00:17:57,040 --> 00:17:58,800
that has flowed through the network,

289
00:17:58,800 --> 00:18:03,200
to finally decide whether it thinks
it's a stop sign, yes or no.

290
00:18:05,920 --> 00:18:07,880
The extraordinary thing
<font color="

291
00:18:07,880 --> 00:18:11,160
is that if you only show it
one picture,

292
00:18:11,160 --> 00:18:14,400
it'll just be guessing
whether it's a stop sign or not.

293
00:18:14,400 --> 00:18:18,200
But show it thousands
and tell it when it's right or wrong

294
00:18:18,200 --> 00:18:22,760
and the AI learns to recognise
the sign itself,

295
00:18:22,760 --> 00:18:26,600
<font color="
every time it makes a mistake.

296
00:18:28,240 --> 00:18:30,400
In the neural networks
you'll find in a car,

297
00:18:30,400 --> 00:18:32,600
they'll be classifying
not just stop signs,

298
00:18:32,600 --> 00:18:37,120
but pedestrians, vehicles,
lampposts, road markings.

299
00:18:37,120 --> 00:18:41,920
They are gigantic
<font color="

300
00:18:41,920 --> 00:18:44,800
but the principle is still the same.

301
00:18:44,800 --> 00:18:48,080
This is a machine built
through trial and error.

302
00:18:55,880 --> 00:19:01,280
But if errors happen
in the real world, on our roads,

303
00:19:01,280 --> 00:19:03,880
the consequences can be fatal.

304
00:19:11,680 --> 00:19:14,080
THUD, TYRES SKID

305
00:19:22,920 --> 00:19:28,600
<font color="
failed to stop at a stop sign

306
00:19:28,600 --> 00:19:31,520
and ploughed through a T junction.

307
00:19:35,600 --> 00:19:37,640
MAN: I'm here. The 911's here.
How are you, sir?

308
00:19:43,720 --> 00:19:44,880
OFFICER: OK.

309
00:19:44,880 --> 00:19:48,800
The driver of the Tesla,
<font color="

310
00:19:48,800 --> 00:19:52,640
had been on his phone
when it fell out of his hand.

311
00:19:52,640 --> 00:19:54,360
He bent down to pick it up,

312
00:19:54,360 --> 00:19:57,920
leaving Tesla's Autopilot
to drive the car.

313
00:19:57,920 --> 00:19:59,680
Check his back. Check his back.

314
00:19:59,680 --> 00:20:01,520
But something went wrong.

315
00:20:03,880 --> 00:20:09,760
<font color="
Dillon Angulo's truck at 62mph.

316
00:20:32,440 --> 00:20:35,800
Remarkably, Dillon survived
the accident.

317
00:20:43,400 --> 00:20:45,960
But he wasn't alone that night.

318
00:21:11,920 --> 00:21:14,440
This picture was actually the day
of the crash.

319
00:21:17,000 --> 00:21:18,040
And...

320
00:21:21,200 --> 00:21:22,320
..we were going fishing,

321
00:21:22,320 --> 00:21:26,240
<font color="
at the bait store and...

322
00:21:27,360 --> 00:21:30,880
At the time of the accident, Dillon
had been with his new girlfriend,

323
00:21:30,880 --> 00:21:33,760
22-year-old Naibel Benavides.

324
00:21:34,760 --> 00:21:37,240
Can I see her?

325
00:21:37,240 --> 00:21:39,000
Oh...

326
00:21:39,000 --> 00:21:42,120
The impact from the Tesla
killed Naibel instantly.

327
00:21:43,440 --> 00:21:45,480
<font color="

328
00:21:45,480 --> 00:21:49,160
Naibel always had this peace
and happiness to her.

329
00:21:49,160 --> 00:21:51,000
And just being around her
would just...

330
00:21:52,720 --> 00:21:54,280
It would rub off on you, you know?

331
00:21:55,640 --> 00:21:57,120
I was going to meet her mom
the next day.

332
00:21:57,120 --> 00:21:59,040
You know,
<font color="

333
00:21:59,040 --> 00:22:02,920
and then I was going to cook lunch
for her mom the next day.

334
00:22:02,920 --> 00:22:05,920
And... Was that the first time
you were going to meet her mum?

335
00:22:05,920 --> 00:22:08,560
Yeah, that was going to be
the first time. Mmm.

336
00:22:08,560 --> 00:22:11,560
<font color="
the first time that...

337
00:22:11,560 --> 00:22:15,440
..I have to meet her mom
were under these circumstances.

338
00:22:22,600 --> 00:22:24,680
Oh, my gosh.

339
00:22:24,680 --> 00:22:26,520
I'm so sorry this happened to you.

340
00:22:31,400 --> 00:22:34,960
Right away, they started doing
an investigation into the accident.

341
00:22:36,000 --> 00:22:40,120
<font color="
on the police body cam video...

342
00:22:42,560 --> 00:22:45,640
..and in that police body cam video,
the driver, yeah, he's like,

343
00:22:45,640 --> 00:22:50,280
"I was driving. I was on the phone.
I had the car on Autopilot Cruise."

344
00:22:51,840 --> 00:22:56,720
I started to do research and I had
<font color="

345
00:22:58,560 --> 00:23:03,200
This thing called Autopilot,
where the cars can drive themselves,

346
00:23:03,200 --> 00:23:04,720
you know? And...

347
00:23:06,840 --> 00:23:08,560
And right then and there,
I was like...

348
00:23:09,960 --> 00:23:12,880
.."This guy was relying on this car
to drive itself.

349
00:23:14,640 --> 00:23:16,400
<font color="

350
00:23:28,240 --> 00:23:31,360
Autopilot is Tesla's advanced

351
00:23:31,360 --> 00:23:32,560
driver assist function,

352
00:23:32,560 --> 00:23:35,360
highlighted in their slick
promotional videos

353
00:23:35,360 --> 00:23:38,360
that sell a vision of
technological sophistication,

354
00:23:38,360 --> 00:23:40,400
safety and convenience,

355
00:23:40,400 --> 00:23:44,680
<font color="
and change lanes on real roads

356
00:23:44,680 --> 00:23:47,160
in the real world.

357
00:23:47,160 --> 00:23:50,360
Tesla Car next year will probably be

358
00:23:50,360 --> 00:23:52,880
90% capable of Autopilot,

359
00:23:52,880 --> 00:23:55,280
like so 90% of your miles
could be on auto.

360
00:23:55,280 --> 00:23:57,760
Other car companies will follow.

361
00:23:57,760 --> 00:23:59,360
<font color="

362
00:23:59,360 --> 00:24:03,360
that his cars can completely
drive themselves,

363
00:24:03,360 --> 00:24:05,280
but they can't.

364
00:24:05,280 --> 00:24:07,720
The driver's manual
says a human still has to be

365
00:24:07,720 --> 00:24:09,760
in full control of the car.

366
00:24:10,880 --> 00:24:14,280
This thing is psychotic.
<font color="

367
00:24:14,280 --> 00:24:16,040
I'm not involved in this.
Watch the road.

368
00:24:16,040 --> 00:24:19,480
What do you think Autopilot...?!
Whoa, Jesus, that was scary.

369
00:24:19,480 --> 00:24:22,640
Wait, there's a double yellow line.
Am I on the right side of the road?

370
00:24:22,640 --> 00:24:26,480
<font color="
Are you serious? Yes!

371
00:24:26,480 --> 00:24:28,920
Look! Don't fucking trust
this thing.

372
00:24:28,920 --> 00:24:33,600
And there's serious safety concerns
over the Autopilot feature.

373
00:24:35,760 --> 00:24:39,040
REPORTER: This Tesla crashed into
<font color="

374
00:24:39,040 --> 00:24:40,280
killing the driver.

375
00:24:40,280 --> 00:24:43,880
Another slammed into
a parked fire truck in Utah.

376
00:24:43,880 --> 00:24:46,800
Both had the Autopilot feature on.

377
00:24:51,640 --> 00:24:54,000
This is the bend right before

378
00:24:54,000 --> 00:24:56,680
the intersection where
the accident happened.

379
00:24:56,680 --> 00:25:00,280
<font color="

380
00:25:00,280 --> 00:25:02,800
you could already see the red light
blinking from right here.

381
00:25:02,800 --> 00:25:05,160
Oh, yeah. I see it.

382
00:25:14,080 --> 00:25:19,200
This is where her body
ended up laying.

383
00:25:19,200 --> 00:25:21,720
You know, we'd pulled over
to look at the stars.

384
00:25:23,120 --> 00:25:27,600
<font color="
you know?

385
00:25:27,600 --> 00:25:31,160
That's how fast the car was going
straight through this intersection.

386
00:25:32,480 --> 00:25:34,720
Six months after
the fatal collision,

387
00:25:34,720 --> 00:25:37,520
the man behind the wheel of
the Tesla, George McGee,

388
00:25:37,520 --> 00:25:42,720
<font color="
which he didn't contest.

389
00:25:44,800 --> 00:25:48,240
But Dillon wanted Tesla
to also be held accountable.

390
00:25:49,240 --> 00:25:51,480
These car manufacturers,
they need to do a better job

391
00:25:51,480 --> 00:25:53,880
with designing these cars.

392
00:25:55,480 --> 00:26:00,360
We want justice. Naibel's family
<font color="

393
00:26:00,360 --> 00:26:01,880
These cars were allowed on the road

394
00:26:01,880 --> 00:26:04,120
before they were ready
to be on the road.

395
00:26:04,120 --> 00:26:05,880
They were not safe.

396
00:26:05,880 --> 00:26:09,840
And they were advertised as these
cars that can drive themselves.

397
00:26:15,080 --> 00:26:18,960
Tesla offered Dillon and
<font color="

398
00:26:18,960 --> 00:26:20,880
to draw a line under the case.

399
00:26:22,400 --> 00:26:26,560
But they refused
and a court date was set.

400
00:26:27,680 --> 00:26:32,960
Tesla would now face a jury trial
over its Autopilot system.

401
00:26:46,880 --> 00:26:48,760
NEWSREADER: First At Five,
the video showing the moments

402
00:26:48,760 --> 00:26:51,840
<font color="
self-driving car crashes.

403
00:26:51,840 --> 00:26:56,600
Backup driver Rafaela Vasquez looks
down for approximately four seconds.

404
00:26:56,600 --> 00:26:59,880
Back in 2018, the fallout
from Rafaela's crash

405
00:26:59,880 --> 00:27:04,120
had left the whole self-driving
industry hanging in the balance.

406
00:27:04,120 --> 00:27:05,720
<font color="

407
00:27:05,720 --> 00:27:09,080
who bears the responsibility
when a car makes a misstep?

408
00:27:09,080 --> 00:27:12,720
Is there enough safety built in
before we deploy these vehicles?

409
00:27:12,720 --> 00:27:15,240
Is there any suggestion
at the moment

410
00:27:15,240 --> 00:27:18,400
that Uber has done something wrong?
<font color="

411
00:27:21,200 --> 00:27:24,320
Under intense media
and political scrutiny,

412
00:27:24,320 --> 00:27:27,800
Tempe's police investigation
left no stone unturned.

413
00:27:29,040 --> 00:27:34,080
So the police have sent over
all of their digital evidence.

414
00:27:34,080 --> 00:27:36,080
Erm...

415
00:27:36,080 --> 00:27:38,680
And I have not yet watched this.
<font color="

416
00:27:40,320 --> 00:27:45,560
The actual car in the police
compound being analysed.

417
00:27:45,560 --> 00:27:47,280
Oh, my gosh, look at that.

418
00:27:47,280 --> 00:27:50,760
And then there's also one
that says "Seize phones."

419
00:27:53,680 --> 00:27:55,560
They're knocking on someone's door.
It's the police department.

420
00:27:55,560 --> 00:27:58,400
<font color="
everyone's faces.

421
00:28:02,080 --> 00:28:04,400
Hey. Hey, Rafaela.

422
00:28:04,400 --> 00:28:07,640
We're coming up to do follow up
regarding the accident,

423
00:28:07,640 --> 00:28:10,480
because we have a seizure warrant
for your cellphone.

424
00:28:10,480 --> 00:28:13,200
They've got a warrant for her phone.

425
00:28:13,200 --> 00:28:15,960
<font color="
did you want? OK, Rafaela.

426
00:28:15,960 --> 00:28:19,120
How many phones do you have?
What? How many phones do you have?

427
00:28:19,120 --> 00:28:21,440
I have my work phone
and then my personal phone.

428
00:28:21,440 --> 00:28:23,920
But my work phone is the one I had
<font color="

429
00:28:23,920 --> 00:28:25,640
So the warrant's
saying we need both phones.

430
00:28:25,640 --> 00:28:27,960
Well, we need all the phones.
How many?

431
00:28:27,960 --> 00:28:29,040
All of your phones.

432
00:28:30,040 --> 00:28:31,680
Why are they taking her phones?

433
00:28:36,040 --> 00:28:37,960
Thank you, Rafaela.

434
00:28:44,440 --> 00:28:46,200
Hi! Hi, good morning.

435
00:28:46,200 --> 00:28:50,000
<font color="
the lead detective on the case.

436
00:28:50,000 --> 00:28:53,600
He agreed to show me the footage
he'd obtained from Uber.

437
00:28:53,600 --> 00:28:57,600
So this is the footage
with all three of the camera views.

438
00:28:57,600 --> 00:28:59,520
When I first hit play,

439
00:28:59,520 --> 00:29:03,440
we can see the driver
<font color="

440
00:29:05,480 --> 00:29:08,680
And there didn't seem to be
a logical explanation

441
00:29:08,680 --> 00:29:10,920
as to why she was looking down.

442
00:29:10,920 --> 00:29:14,280
So we're coming up to the crash now,
are we? Yes.

443
00:29:14,280 --> 00:29:15,320
And there.

444
00:29:17,560 --> 00:29:19,360
<font color="
in this? Of course.

445
00:29:19,360 --> 00:29:22,480
I just want to see what's happening
immediately before. Sure.

446
00:29:24,120 --> 00:29:25,880
So looking down, looking down,
looking down,

447
00:29:25,880 --> 00:29:27,320
looking down, looking down,
looking down,

448
00:29:27,320 --> 00:29:30,280
<font color="
HANNAH GASPS

449
00:29:30,280 --> 00:29:32,320
What does she say
she was looking at?

450
00:29:32,320 --> 00:29:35,400
Initially, there was a statement

451
00:29:35,400 --> 00:29:40,400
that it had to do with the iPad
in the centre of the vehicle.

452
00:29:40,400 --> 00:29:41,880
And does that broadly stack up?

453
00:29:41,880 --> 00:29:46,000
<font color="
at screens while they were driving?

454
00:29:46,000 --> 00:29:49,520
So the... Yes, that was... One
of the purposes of her job,

455
00:29:49,520 --> 00:29:51,400
was to monitor the vehicle.

456
00:29:51,400 --> 00:29:52,920
And if there was anything abnormal,

457
00:29:52,920 --> 00:29:55,200
she would need to interact
with that screen.

458
00:29:55,200 --> 00:29:57,520
<font color="
instructions, right,

459
00:29:57,520 --> 00:30:00,240
that, like... Yes. ..they're
supposed to be looking at screens

460
00:30:00,240 --> 00:30:02,560
but simultaneously monitoring
the road.

461
00:30:02,560 --> 00:30:04,360
Yeah, it definitely presents
<font color="

462
00:30:04,360 --> 00:30:07,560
Yeah. However, when we of course
looked at what the screen

463
00:30:07,560 --> 00:30:09,760
was actually showing at the time,

464
00:30:09,760 --> 00:30:11,960
it didn't show
that there was any sort of alerts

465
00:30:11,960 --> 00:30:14,240
or any sort of interaction
with the screen.

466
00:30:14,240 --> 00:30:15,800
<font color="

467
00:30:15,800 --> 00:30:20,040
We saw the apps that were installed
on her personal phone.

468
00:30:20,040 --> 00:30:23,160
We noticed Hulu was an active app
on her phone,

469
00:30:23,160 --> 00:30:25,160
and when we wrote
a search warrant to the company,

470
00:30:25,160 --> 00:30:28,680
they responded with a printout
<font color="

471
00:30:28,680 --> 00:30:32,320
And that's what gave us probably
the most important information

472
00:30:32,320 --> 00:30:34,120
at the time.
And what did it say?

473
00:30:35,320 --> 00:30:38,440
So this was the information
that was provided from Hulu.

474
00:30:38,440 --> 00:30:42,560
And it shows that the show The Voice
<font color="

475
00:30:42,560 --> 00:30:44,360
to her personal phone.

476
00:30:44,360 --> 00:30:47,400
So what was your take
on seeing all of this?

477
00:30:47,400 --> 00:30:49,480
Essentially, this goes to
the beginning of her shift.

478
00:30:49,480 --> 00:30:53,720
She started her shift by driving
the vehicle out of the garage,

479
00:30:53,720 --> 00:30:56,120
<font color="
she even got onto the roadway,

480
00:30:56,120 --> 00:30:59,040
that she had set up
and started streaming the Hulu.

481
00:30:59,040 --> 00:31:03,400
And so I think that the conscious
decision to provide yourself

482
00:31:03,400 --> 00:31:07,480
with a likely distraction
before even getting onto the roadway

483
00:31:07,480 --> 00:31:11,600
<font color="
to be made.

484
00:31:22,960 --> 00:31:27,640
So I found... I found a couple
of clips around the time.

485
00:31:27,640 --> 00:31:30,560
And lots of them
are not on your side.

486
00:31:30,560 --> 00:31:33,040
Some of them were terrible,
especially at first.

487
00:31:33,040 --> 00:31:35,280
Take a look at this.
<font color="

488
00:31:35,280 --> 00:31:39,400
It turns out she was streaming
that singing competition The Voice

489
00:31:39,400 --> 00:31:42,280
at the time when she should have
been watching the road.

490
00:31:42,280 --> 00:31:46,000
Rafaela Vasquez was riding
in autonomous mode at the time,

491
00:31:46,000 --> 00:31:49,760
but she was distracted
<font color="

492
00:31:49,760 --> 00:31:53,520
of the nearly 22 minutes
before the crash.

493
00:31:53,520 --> 00:31:55,480
Right there - they're right.

494
00:31:55,480 --> 00:31:58,480
I wasn't distracted,
I was doing my job.

495
00:31:58,480 --> 00:32:00,600
I had other things to do
other than operate the vehicle,

496
00:32:00,600 --> 00:32:01,880
<font color="

497
00:32:01,880 --> 00:32:03,800
So I had duties assigned.

498
00:32:03,800 --> 00:32:06,640
Like, in the video, you see me
look away. I am looking away.

499
00:32:06,640 --> 00:32:09,280
But if you time them,
and even the police did,

500
00:32:09,280 --> 00:32:11,000
I never looked away
for more than five seconds.

501
00:32:11,000 --> 00:32:12,280
<font color="

502
00:32:12,280 --> 00:32:13,720
We were trained to look, boom,

503
00:32:13,720 --> 00:32:16,280
iPad five seconds,
then look back, boom.

504
00:32:16,280 --> 00:32:18,440
Over here, five seconds,
look back, boom.

505
00:32:18,440 --> 00:32:23,200
Our cellphones, we have to
Bluetooth them in to the vehicle.

506
00:32:23,200 --> 00:32:26,040
<font color="
I was listening to it.

507
00:32:26,040 --> 00:32:27,760
But the police said I was watching -

508
00:32:27,760 --> 00:32:29,520
that means that's
a definitive statement.

509
00:32:29,520 --> 00:32:31,040
You just told the public

510
00:32:31,040 --> 00:32:33,600
that you have evidence
that I was watching something.

511
00:32:33,600 --> 00:32:35,320
<font color="

512
00:32:35,320 --> 00:32:38,080
You have evidence that I was
Bluetooth streaming something,

513
00:32:38,080 --> 00:32:41,760
but, hello, streaming, Bluetooth -
you can listen to stuff.

514
00:32:41,760 --> 00:32:43,880
People do it all the time
with music.

515
00:32:43,880 --> 00:32:48,760
Everyone automatically assumed -
<font color="

516
00:32:48,760 --> 00:32:54,520
that I was watching Hulu, and
because of that, I was negligent,

517
00:32:54,520 --> 00:32:56,600
therefore, I was unable
to prevent the accident,

518
00:32:56,600 --> 00:32:58,840
therefore the charge.

519
00:32:58,840 --> 00:33:02,440
The investigation lasted
for two and a half years

520
00:33:02,440 --> 00:33:05,200
<font color="

521
00:33:05,200 --> 00:33:07,400
Where were you
when the indictment came through?

522
00:33:07,400 --> 00:33:09,800
I was at home.
My attorneys informed me.

523
00:33:09,800 --> 00:33:11,600
They just told me that
I was going to be indicted

524
00:33:11,600 --> 00:33:12,680
for negligent homicide.

525
00:33:12,680 --> 00:33:15,520
I said, "What?"
<font color="

526
00:33:15,520 --> 00:33:18,600
Negligent homicide?!
I said, "Homicide?"

527
00:33:18,600 --> 00:33:20,400
I said, "So, murder?"

528
00:33:20,400 --> 00:33:24,160
That's just me,
how I interpret that. So I now...

529
00:33:24,160 --> 00:33:27,280
I'm like, "Are you k..."
I didn't know what to say or do.

530
00:33:28,520 --> 00:33:29,640
Now I'm in shock.

531
00:33:31,000 --> 00:33:35,760
<font color="
to eight years in prison.

532
00:33:42,720 --> 00:33:45,160
With conflicting accounts
from both sides,

533
00:33:45,160 --> 00:33:48,720
I decided to track down
the official accident report.

534
00:33:53,040 --> 00:33:55,800
The report concluded that the crash
was probably caused

535
00:33:55,800 --> 00:34:00,400
<font color="
the driving environment,

536
00:34:00,400 --> 00:34:03,240
but it was also critical of Uber,

537
00:34:03,240 --> 00:34:07,000
uncovering troubling flaws
in the design of its AI programming.

538
00:34:10,720 --> 00:34:12,600
This table
is particularly interesting,

539
00:34:12,600 --> 00:34:15,320
cos this is like seeing
<font color="

540
00:34:15,320 --> 00:34:19,320
of what the driverless car
was thinking at the time.

541
00:34:19,320 --> 00:34:22,560
The car actually notices that
there's something there

542
00:34:22,560 --> 00:34:25,320
5.6 seconds before the collision,

543
00:34:25,320 --> 00:34:30,160
which is plenty of time
to start braking or turning.

544
00:34:30,160 --> 00:34:32,520
<font color="

545
00:34:32,520 --> 00:34:34,920
that detects there's something
in the distance.

546
00:34:34,920 --> 00:34:37,120
It predicts it's a vehicle, though.

547
00:34:37,120 --> 00:34:39,360
And then 0.4 seconds later,

548
00:34:39,360 --> 00:34:43,000
the lidar kicks in and picks up
that there's something there.

549
00:34:44,600 --> 00:34:45,960
The lidar doesn't know
<font color="

550
00:34:45,960 --> 00:34:49,560
It just classifies it as "other",
something unknown.

551
00:34:49,560 --> 00:34:53,000
Now the lidar
keeps changing its mind.

552
00:34:53,000 --> 00:34:55,720
So a full second later,

553
00:34:55,720 --> 00:34:57,800
changes its mind,
decides it's a vehicle,

554
00:34:57,800 --> 00:35:01,960
then 0.3 of a second later,
<font color="

555
00:35:01,960 --> 00:35:04,360
and then it keeps switching -

556
00:35:04,360 --> 00:35:06,960
vehicle, other, vehicle, other.

557
00:35:06,960 --> 00:35:12,280
The computer within this driverless
car is not connecting the dots.

558
00:35:12,280 --> 00:35:17,840
It's not seeing this as one object
whose classification keeps changing,

559
00:35:17,840 --> 00:35:21,200
whose path is slowly moving
<font color="

560
00:35:21,200 --> 00:35:25,040
Instead, it is seeing this as
brand-new objects every single time.

561
00:35:30,520 --> 00:35:34,200
In fact, it's only 1.2 seconds
before impact

562
00:35:34,200 --> 00:35:37,000
that it finally decides to settle
that it's a bicycle.

563
00:35:41,320 --> 00:35:43,920
Obviously way too late
<font color="

564
00:35:45,880 --> 00:35:47,480
Terrifying.

565
00:35:47,480 --> 00:35:51,760
You design a system that fails
to track the path of an object

566
00:35:51,760 --> 00:35:54,280
until it's decided what it is?

567
00:35:54,280 --> 00:35:56,800
If you've got something that
you're going to collide with,

568
00:35:56,800 --> 00:36:00,040
I don't care what it is! I care
<font color="

569
00:36:05,560 --> 00:36:06,640
That's insane.

570
00:36:08,080 --> 00:36:10,680
"Although the system sensed
the pedestrian nearly six seconds

571
00:36:10,680 --> 00:36:15,120
"before the impact, the system
never classified her as a pedestrian

572
00:36:15,120 --> 00:36:19,000
"or predicted correctly her goal.

573
00:36:19,000 --> 00:36:21,400
"The system design did not include

574
00:36:21,400 --> 00:36:24,720
<font color="
for jaywalking pedestrians."

575
00:36:26,720 --> 00:36:30,120
It wasn't designed
to recognise people

576
00:36:30,120 --> 00:36:32,040
unless they were on a crosswalk.

577
00:36:36,360 --> 00:36:38,200
I think this is
pretty damning, actually.

578
00:36:41,280 --> 00:36:43,240
What was this thing doing
on the roads?

579
00:36:52,000 --> 00:36:54,800
<font color="
about what was happening

580
00:36:54,800 --> 00:36:59,880
inside Uber around the time
of Rafaela's crash in 2018.

581
00:37:01,200 --> 00:37:04,200
After some digging,
I found Robbie Miller...

582
00:37:04,200 --> 00:37:06,120
Robbie, hi!

583
00:37:06,120 --> 00:37:08,440
..an operations manager
who'd been at Uber

584
00:37:08,440 --> 00:37:11,040
<font color="

585
00:37:11,040 --> 00:37:13,000
He had quit on safety grounds

586
00:37:13,000 --> 00:37:15,920
just a few days before
the fatal collision.

587
00:37:16,960 --> 00:37:19,080
I'd been working
in the self-driving space

588
00:37:19,080 --> 00:37:21,560
for four or five years
at this point.

589
00:37:21,560 --> 00:37:25,120
I was running the self-driving
<font color="

590
00:37:25,120 --> 00:37:29,440
Eventually, there was a move
to combine the operations

591
00:37:29,440 --> 00:37:34,680
of the testing for self-driving cars
and self-driving trucks.

592
00:37:36,640 --> 00:37:37,880
How did you feel about that?

593
00:37:37,880 --> 00:37:40,040
I was not at all comfortable
with it.

594
00:37:40,040 --> 00:37:42,960
The self-driving cars
<font color="

595
00:37:42,960 --> 00:37:48,320
There's a lot of really scary
incidences that are occurring -

596
00:37:48,320 --> 00:37:50,720
near misses, near collisions.

597
00:37:50,720 --> 00:37:53,200
We would have, you know,
an incident

598
00:37:53,200 --> 00:37:58,560
where the car was driving on
the sidewalk...in broad daylight.

599
00:37:58,560 --> 00:38:00,880
<font color="

600
00:38:00,880 --> 00:38:03,800
They are headed on a path

601
00:38:03,800 --> 00:38:07,440
where someone is going to get
seriously injured or worse.

602
00:38:07,440 --> 00:38:10,280
And so I...

603
00:38:10,280 --> 00:38:12,000
I gave notice.

604
00:38:12,000 --> 00:38:16,440
There's this overarching fear,
I would say, at Uber,

605
00:38:16,440 --> 00:38:21,320
that Waymo is about to release
<font color="

606
00:38:21,320 --> 00:38:23,040
And it's...

607
00:38:23,040 --> 00:38:27,480
I think it's very scary for Uber's
leadership to not have a response.

608
00:38:27,480 --> 00:38:31,680
So it's turned into a race?
It is absolutely a race.

609
00:38:31,680 --> 00:38:35,600
Waymo, owned by Google's
parent company Alphabet,

610
00:38:35,600 --> 00:38:38,440
<font color="

611
00:38:38,440 --> 00:38:42,840
Just five months after Waymo
launched its first public trial,

612
00:38:42,840 --> 00:38:46,160
Uber moved from two operators
in the car to one.

613
00:38:48,520 --> 00:38:52,720
You need to show to your investors,
"Hey, we're making this progress."

614
00:38:52,720 --> 00:38:55,680
An easy way to do that
<font color="

615
00:38:55,680 --> 00:38:57,720
And this is something I mentioned -

616
00:38:57,720 --> 00:38:59,200
"You need that second person
in the vehicle.

617
00:38:59,200 --> 00:39:02,960
"You're not ready to take
that person out of the vehicle."

618
00:39:02,960 --> 00:39:06,920
Just a few days after I left,
the crash occurred.

619
00:39:08,040 --> 00:39:10,200
<font color="
about it?

620
00:39:11,720 --> 00:39:13,280
I was driving

621
00:39:13,280 --> 00:39:19,720
and I received a phone call from
one of my former co-workers at Uber.

622
00:39:21,760 --> 00:39:25,920
And I sat in the parking lot
and cried for 15 minutes.

623
00:39:31,960 --> 00:39:33,600
Wow.

624
00:39:33,600 --> 00:39:36,600
<font color="
the technology.

625
00:39:36,600 --> 00:39:42,800
I believe in the technology.
I want the technology to succeed.

626
00:39:42,800 --> 00:39:44,320
But there's...

627
00:39:44,320 --> 00:39:46,040
There's a way to do it.

628
00:39:54,120 --> 00:39:57,040
A few years after
Rafaela's fatal crash,

629
00:39:57,040 --> 00:40:00,000
Uber sold off
<font color="

630
00:40:00,000 --> 00:40:02,240
leaving the path clear for Waymo.

631
00:40:05,920 --> 00:40:09,440
Here you go! Waymo's just crashed.

632
00:40:09,440 --> 00:40:12,080
AI technology - not working!

633
00:40:13,240 --> 00:40:15,960
Why is this happening to me
on a Monday? I'm in a Waymo car...

634
00:40:15,960 --> 00:40:17,800
AUTOMATED VOICE: Connected to
<font color="

635
00:40:17,800 --> 00:40:20,240
..and... This call may be recorded
for quality assurance.

636
00:40:20,240 --> 00:40:22,320
..this car is just going in circles.

637
00:40:22,320 --> 00:40:24,960
Hi there, Mike. This is Gab.
I'm calling from Waymo support...

638
00:40:24,960 --> 00:40:26,800
Yeah, I got a flight to catch.

639
00:40:26,800 --> 00:40:29,320
<font color="
I'm getting dizzy.

640
00:40:30,800 --> 00:40:35,040
Waymo are now the dominant force
in driverless taxis in the US,

641
00:40:35,040 --> 00:40:38,080
with 2,500 of them on the road.

642
00:40:42,680 --> 00:40:45,560
Experts have praised
their safety record.

643
00:40:48,360 --> 00:40:52,800
But some see these cars as symbols
<font color="

644
00:40:52,800 --> 00:40:56,560
and their disconnection from
the lives of ordinary people.

645
00:40:56,560 --> 00:40:58,000
Hey! Hello.

646
00:40:59,040 --> 00:41:02,680
And there's one group
who've decided to take action.

647
00:41:02,680 --> 00:41:05,400
Nice to meet you. It's nice
<font color="

648
00:41:05,400 --> 00:41:06,480
Doing well. Doing well.

649
00:41:06,480 --> 00:41:09,640
They call themselves
the Safe Street Rebels

650
00:41:09,640 --> 00:41:12,960
and carry out
their operations incognito.

651
00:41:12,960 --> 00:41:18,200
What is it about the autonomous
driving that you dislike, though?

652
00:41:18,200 --> 00:41:20,480
They are heavily
<font color="

653
00:41:20,480 --> 00:41:23,520
as the future of public transit,

654
00:41:23,520 --> 00:41:25,160
and we just fundamentally
don't think

655
00:41:25,160 --> 00:41:26,600
they're the future of
public transit.

656
00:41:26,600 --> 00:41:29,680
They're just a taxi
where you don't talk to someone.

657
00:41:29,680 --> 00:41:33,080
And when that car is not parked,
<font color="

658
00:41:33,080 --> 00:41:36,000
50% of the miles they drive,
there's nobody in the vehicle.

659
00:41:36,000 --> 00:41:38,520
In addition, they cannot be ticketed

660
00:41:38,520 --> 00:41:41,080
for any kind of moving violation
in the city.

661
00:41:41,080 --> 00:41:43,120
We have videos of them driving 40mph

662
00:41:43,120 --> 00:41:46,240
on the wrong side of the road,
<font color="

663
00:41:46,240 --> 00:41:47,480
They're completely immune.

664
00:41:47,480 --> 00:41:49,520
What, they're above
the kind of rules that would stand

665
00:41:49,520 --> 00:41:51,200
for an Uber driver or a Lyft driver?

666
00:41:51,200 --> 00:41:54,440
Yes, and if they can't do something
about it, we can.

667
00:41:55,640 --> 00:41:57,360
To express their frustration,

668
00:41:57,360 --> 00:42:01,160
<font color="
disabling Waymos.

669
00:42:04,120 --> 00:42:05,600
Let me understand
the strategy, then.

670
00:42:05,600 --> 00:42:08,480
So how easy are they to override?

671
00:42:08,480 --> 00:42:11,560
LAUGHS: Pretty easy!
Definitely pretty easy.

672
00:42:11,560 --> 00:42:14,880
We're not allowed to show you
<font color="

673
00:42:14,880 --> 00:42:17,680
although it's not exactly high-tech.

674
00:42:17,680 --> 00:42:19,920
So you're not putting them
out of action permanently? No, no.

675
00:42:19,920 --> 00:42:21,720
And we're not causing any damage
to them either.

676
00:42:21,720 --> 00:42:22,760
It's like... It's painter's tape,

677
00:42:22,760 --> 00:42:24,960
<font color="
when you remove it.

678
00:42:24,960 --> 00:42:27,480
There's one...
There's one coming. Right.

679
00:42:27,480 --> 00:42:30,320
There's one, there's one...
Yeah. Check this one.

680
00:42:30,320 --> 00:42:31,360
Someone go in front.

681
00:42:35,880 --> 00:42:38,560
MAN: Fuck it up!
<font color="

682
00:42:38,560 --> 00:42:40,120
CHEERING

683
00:42:40,120 --> 00:42:41,200
Did you hear that?

684
00:42:41,200 --> 00:42:44,480
There was people on the sidewalk
who were, like, cheering them on.

685
00:42:45,760 --> 00:42:48,720
The disabled car was now blocking
the road.

686
00:42:48,720 --> 00:42:51,240
CHIME
AUTOMATED VOICE: Please step back.

687
00:42:53,280 --> 00:42:57,840
<font color="
but is stuck.

688
00:42:57,840 --> 00:42:59,880
There's another one coming.
Oh, my gosh!

689
00:43:02,240 --> 00:43:03,280
Three of them...

690
00:43:04,880 --> 00:43:06,440
..and they're just stuck.

691
00:43:06,440 --> 00:43:09,200
And now there's another car
behind, flashing.

692
00:43:09,200 --> 00:43:11,800
But it has a human driver,
<font color="

693
00:43:15,160 --> 00:43:17,800
Is it actually illegal,
what you're doing?

694
00:43:17,800 --> 00:43:18,880
We don't think it's illegal.

695
00:43:18,880 --> 00:43:20,320
We can't find any law
that it breaks.

696
00:43:20,320 --> 00:43:23,640
It's not vandalism, so... It's hard
to point any law we're breaking.

697
00:43:25,360 --> 00:43:28,400
<font color="
could do this? I mean, can you...

698
00:43:28,400 --> 00:43:31,440
I don't know, is there, like,
not a more sort of democratic way?

699
00:43:31,440 --> 00:43:32,760
It would be nice if we could vote,

700
00:43:32,760 --> 00:43:34,320
if we had the opportunity
to vote on them,

701
00:43:34,320 --> 00:43:36,960
but we have not been presented
<font color="

702
00:43:36,960 --> 00:43:39,000
Is this... Does it feel a bit like
this is something

703
00:43:39,000 --> 00:43:41,240
that has happened to you
rather than with you?

704
00:43:41,240 --> 00:43:42,760
I mean, absolutely.

705
00:43:42,760 --> 00:43:44,040
Hey, yo, there's one coming
the other side.

706
00:43:44,040 --> 00:43:46,840
Other side, other side, other side.
<font color="

707
00:43:49,800 --> 00:43:51,320
Yeah. You want to get in front?

708
00:43:57,040 --> 00:43:59,600
They are surprisingly easy to bully,
aren't they?

709
00:44:01,720 --> 00:44:03,960
The other thing that
I think was really surprising

710
00:44:03,960 --> 00:44:08,200
was just how many of them
were going past, right?

711
00:44:08,200 --> 00:44:11,560
<font color="
but, I mean...

712
00:44:11,560 --> 00:44:14,040
..hardly any of them
have people inside.

713
00:44:17,760 --> 00:44:19,160
Yeah, it's empty.

714
00:44:23,320 --> 00:44:26,000
They did say something
that I hadn't considered before,

715
00:44:26,000 --> 00:44:29,400
which is about how they are
continually circling

716
00:44:29,400 --> 00:44:31,480
<font color="
people inside them.

717
00:44:31,480 --> 00:44:33,440
And that I hadn't considered,
because of course,

718
00:44:33,440 --> 00:44:36,200
there's a congestion aspect...

719
00:44:36,200 --> 00:44:37,760
..but there's an
environmental aspect, too.

720
00:44:37,760 --> 00:44:39,480
I mean, they've gotta be powered.

721
00:44:41,360 --> 00:44:43,360
Hmm. Interesting.

722
00:44:44,360 --> 00:44:45,680
<font color="

723
00:45:09,360 --> 00:45:10,880
It happened fast.

724
00:45:10,880 --> 00:45:13,200
The hearing for
Rafaela Vasquez was scheduled

725
00:45:13,200 --> 00:45:14,680
as a settlement conference,

726
00:45:14,680 --> 00:45:17,400
but it quickly led to a plea deal.

727
00:45:17,400 --> 00:45:21,840
In July 2023, after five years
of legal wrangling,

728
00:45:21,840 --> 00:45:24,840
Rafaela accepted a plea deal
<font color="

729
00:45:24,840 --> 00:45:27,640
The agreement indicates
that you wish to plead guilty to

730
00:45:27,640 --> 00:45:29,120
the crime of endangerment.

731
00:45:29,120 --> 00:45:31,640
The offence is non-dangerous,
non-repetitive.

732
00:45:31,640 --> 00:45:35,760
Is that the crime that you wish
to plead guilty to? Yes.

733
00:45:35,760 --> 00:45:39,680
<font color="
a reduced charge - endangerment -

734
00:45:39,680 --> 00:45:42,720
with the guarantee it would be
lowered to the least-serious

735
00:45:42,720 --> 00:45:46,080
category of offence, following
three years of probation.

736
00:45:49,080 --> 00:45:51,160
Ah! Hey, how are you?
Look who it is. Hi.

737
00:45:51,160 --> 00:45:54,760
<font color="
Al Morrison....

738
00:45:54,760 --> 00:45:56,720
How are you?
..and Marci Kratter.

739
00:45:56,720 --> 00:45:59,120
So good to see you.

740
00:45:59,120 --> 00:46:01,000
Why did you take this case on?

741
00:46:01,000 --> 00:46:03,800
Our job is to fight for
the little guy - no offence -

742
00:46:03,800 --> 00:46:06,680
but she was the little guy.

743
00:46:06,680 --> 00:46:09,080
<font color="

744
00:46:09,080 --> 00:46:12,600
the more we realised just
how one-sided it was.

745
00:46:12,600 --> 00:46:15,520
Where do you think this came from,
then? I mean, is this...

746
00:46:15,520 --> 00:46:18,640
Is this the police,
or is this from Uber?

747
00:46:18,640 --> 00:46:20,560
Uber. Really?

748
00:46:20,560 --> 00:46:24,280
<font color="
to make it not their fault.

749
00:46:24,280 --> 00:46:27,240
And it's David and Goliath. Yeah.

750
00:46:27,240 --> 00:46:29,200
We have this single person

751
00:46:29,200 --> 00:46:34,720
up against a multimillion-dollar
company with unlimited resources.

752
00:46:34,720 --> 00:46:37,720
The problem was that there
<font color="

753
00:46:37,720 --> 00:46:39,360
these cars were programmed

754
00:46:39,360 --> 00:46:41,760
that didn't account for
real-world situations.

755
00:46:41,760 --> 00:46:44,600
The most obvious one is,
the thing wasn't programmed

756
00:46:44,600 --> 00:46:46,440
to deal with jaywalkers.

757
00:46:46,440 --> 00:46:50,400
To save yourself in a campus,
<font color="

758
00:46:50,400 --> 00:46:52,920
to not programme vehicles
to deal with jaywalking

759
00:46:52,920 --> 00:46:54,760
I think is the height of negligence.

760
00:46:54,760 --> 00:46:57,400
I was scared to go to trial.

761
00:46:57,400 --> 00:47:01,840
If I lose, it's prison time,
no ifs, ands or buts.

762
00:47:01,840 --> 00:47:05,080
And the media already
destroyed me out there.

763
00:47:05,080 --> 00:47:07,520
<font color="
of our minds as lawyers,

764
00:47:07,520 --> 00:47:10,960
and certainly our clients' minds,
that it's the exposure.

765
00:47:10,960 --> 00:47:14,240
"What's going to happen to me
if I go to trial and lose?"

766
00:47:14,240 --> 00:47:16,640
But Marci and I felt like
we had a great case,

767
00:47:16,640 --> 00:47:19,680
<font color="
the risk was all hers.

768
00:47:19,680 --> 00:47:22,480
Me not going to trial has
no reflection on them.

769
00:47:22,480 --> 00:47:24,680
These two saved my life.

770
00:47:24,680 --> 00:47:27,600
They absolutely saved my life.

771
00:47:32,320 --> 00:47:35,760
No criminal charges were
ever brought against Uber.

772
00:47:45,640 --> 00:47:48,920
Rafaela got dealt
<font color="

773
00:47:48,920 --> 00:47:51,720
and this didn't play out fairly.

774
00:47:53,520 --> 00:47:55,680
Maybe she was watching her phone.

775
00:47:55,680 --> 00:47:58,160
Maybe she wasn't. Like, I...

776
00:47:58,160 --> 00:48:00,880
Honestly, I don't know,
but I also...

777
00:48:00,880 --> 00:48:04,040
I don't think that
that's the point of this.

778
00:48:04,040 --> 00:48:09,560
<font color="
got this massive corporate,

779
00:48:09,560 --> 00:48:15,240
multibillion-dollar project
which is not prioritising

780
00:48:15,240 --> 00:48:18,920
the safety of the people who are
in the cars, or, like,

781
00:48:18,920 --> 00:48:21,400
the members of the public
who haven't even agreed

782
00:48:21,400 --> 00:48:23,840
<font color="

783
00:48:23,840 --> 00:48:29,440
It is not enough to say that all of
the responsibility lies with

784
00:48:29,440 --> 00:48:31,120
the person who was in the car.

785
00:48:32,280 --> 00:48:35,360
That's not enough.
That's not enough for me.

786
00:48:47,160 --> 00:48:51,440
Back in Miami, Dillon and Naibel's
family were taking Tesla to court

787
00:48:51,440 --> 00:48:55,080
<font color="
that had killed Naibel.

788
00:48:55,080 --> 00:48:59,400
Nice to meet you. I'm Hannah. Adam.
Great to see you. Great to meet you.

789
00:48:59,400 --> 00:49:01,600
What's up, buddy?

790
00:48:59,400 --> 00:49:01,600
SHE CHUCKLES

791
00:49:03,320 --> 00:49:05,280
Dillon’s attorney, Adam Boumel,

792
00:49:05,280 --> 00:49:08,040
<font color="

793
00:49:08,040 --> 00:49:11,440
This is a sort of situation
where there's no precedent at all.

794
00:49:11,440 --> 00:49:14,840
Does that make it quite difficult
to build a legal case when you're,

795
00:49:14,840 --> 00:49:17,680
I don't know, forging a new path?
Extremely. Yeah?

796
00:49:17,680 --> 00:49:19,680
<font color="
from the start.

797
00:49:19,680 --> 00:49:21,960
This is creating the law.

798
00:49:21,960 --> 00:49:25,320
Obviously, from day one,
Tesla's position was,

799
00:49:25,320 --> 00:49:28,560
this is 100% driver's fault.

800
00:49:28,560 --> 00:49:31,680
The thing is that it's difficult
to untangle the responsibility

801
00:49:31,680 --> 00:49:34,640
<font color="
saying that the driver had

802
00:49:34,640 --> 00:49:37,120
no responsibility at all.
No, of course not.

803
00:49:37,120 --> 00:49:41,800
We 100% acknowledge that
the driver had responsibility.

804
00:49:41,800 --> 00:49:45,000
And our position from day one
has been that this is a case

805
00:49:45,000 --> 00:49:46,760
<font color="

806
00:49:46,760 --> 00:49:48,480
Yes, the driver was at fault.

807
00:49:48,480 --> 00:49:52,120
He was distracted, he was
disengaged from the driving task.

808
00:49:52,120 --> 00:49:53,960
But then you have to ask yourself,

809
00:49:53,960 --> 00:49:56,200
why was he disengaged from
the driving task?

810
00:49:56,200 --> 00:50:00,760
And you realise that Tesla
<font color="

811
00:50:00,760 --> 00:50:05,080
this trust of the system
that was unwarranted.

812
00:50:05,080 --> 00:50:10,040
He thought that the car would stop
or swerve or do something

813
00:50:10,040 --> 00:50:12,600
before ploughing into
a parked vehicle.

814
00:50:12,600 --> 00:50:14,600
Do you have the data from
inside the car?

815
00:50:14,600 --> 00:50:16,480
<font color="

816
00:50:16,480 --> 00:50:20,240
So, we were able to finally get
the data from inside the car,

817
00:50:20,240 --> 00:50:24,920
showing what the Autopilot computer
was detecting and processing.

818
00:50:26,800 --> 00:50:29,520
The computer understood that
the car was speeding

819
00:50:29,520 --> 00:50:31,000
towards the end of the road.

820
00:50:32,160 --> 00:50:34,880
<font color="

821
00:50:34,880 --> 00:50:36,920
It appreciated
the blinking red light.

822
00:50:38,120 --> 00:50:40,960
It appreciated Dillon's parked SUV.

823
00:50:43,400 --> 00:50:46,800
So, it identified a lot of things,
and it did nothing.

824
00:50:55,720 --> 00:51:00,080
The car knew what was going on
in front of it,

825
00:51:00,080 --> 00:51:03,800
<font color="
the driver to stop the car.

826
00:51:03,800 --> 00:51:06,760
So, this isn't a case
of misinformation

827
00:51:06,760 --> 00:51:08,440
inside the car's system.

828
00:51:08,440 --> 00:51:11,400
The car has all of the information
it needed in order

829
00:51:11,400 --> 00:51:13,120
to avoid the collision.

830
00:51:13,120 --> 00:51:18,600
<font color="
in the way that the driver thought

831
00:51:18,600 --> 00:51:20,480
it was programmed to behave.

832
00:51:20,480 --> 00:51:24,440
Did the car behave as people
expected and believed it should,

833
00:51:24,440 --> 00:51:29,600
based off of Tesla's marketing
and Elon Musk's statements,

834
00:51:29,600 --> 00:51:33,800
in kind of hyping up
<font color="

835
00:51:33,800 --> 00:51:38,000
When you take that backdrop
and you put it against this case,

836
00:51:38,000 --> 00:51:42,920
the car did nothing that anybody
thought it would or should.

837
00:51:42,920 --> 00:51:45,840
So, this has almost become
a case of, like,

838
00:51:45,840 --> 00:51:50,000
misadvertising or, like,
<font color="

839
00:51:50,000 --> 00:51:53,480
They make the consumers and
drivers believe that

840
00:51:53,480 --> 00:51:55,200
the car is more capable,

841
00:51:55,200 --> 00:51:58,000
creating false expectations
in their drivers.

842
00:51:59,480 --> 00:52:02,240
In the Uber crash,
the blame had been laid

843
00:52:02,240 --> 00:52:04,440
firmly on Rafaela's shoulders.

844
00:52:07,400 --> 00:52:11,120
<font color="
the first time the car itself

845
00:52:11,120 --> 00:52:13,080
would be considered at fault?

846
00:52:33,880 --> 00:52:38,560
Tonight, a Florida jury forcing
Tesla to pay $243 million

847
00:52:38,560 --> 00:52:41,520
to victims in a deadly 2019 crash,

848
00:52:41,520 --> 00:52:45,080
the jury finding flaws
in Tesla's self-driving software

849
00:52:45,080 --> 00:52:46,800
<font color="

850
00:52:46,800 --> 00:52:49,160
We can be proud that we stood up,

851
00:52:49,160 --> 00:52:53,720
and that we did everything in
our power to help shine light

852
00:52:53,720 --> 00:52:55,120
on what's going on.

853
00:52:57,040 --> 00:53:00,960
Against all odds,
Dillon emerged victorious in court,

854
00:53:00,960 --> 00:53:04,160
forcing Tesla to take
<font color="

855
00:53:04,160 --> 00:53:05,960
for the first time ever -

856
00:53:05,960 --> 00:53:08,560
for a crash involving
its Autopilot system.

857
00:53:12,440 --> 00:53:15,880
It was a shocking landmark verdict

858
00:53:15,880 --> 00:53:19,440
that could well reshape
the future of self-driving cars.

859
00:53:20,840 --> 00:53:23,920
When they gave the verdict,
how was it?

860
00:53:23,920 --> 00:53:26,160
<font color="

861
00:53:26,160 --> 00:53:29,720
I mean, like, me and my dad,
we started hugging each other and,

862
00:53:29,720 --> 00:53:34,400
you know, praying...
that we would get justice.

863
00:53:34,400 --> 00:53:36,720
You know, from the beginning,
we knew this was

864
00:53:36,720 --> 00:53:38,080
a joint liability case,

865
00:53:38,080 --> 00:53:42,640
<font color="
Tesla accountable.

866
00:53:42,640 --> 00:53:44,480
And I'm grateful for that.

867
00:53:44,480 --> 00:53:48,640
And I'm grateful that people
heard all the evidence,

868
00:53:48,640 --> 00:53:51,680
and saw that Tesla made a mistake.

869
00:53:51,680 --> 00:53:54,280
I really get the impression
that this has never been

870
00:53:54,280 --> 00:53:55,800
<font color="

871
00:53:55,800 --> 00:53:59,840
It was more important for us
to shine light,

872
00:53:59,840 --> 00:54:04,040
and to show the world that
this technology is not safe.

873
00:54:05,960 --> 00:54:10,280
People will come to me and,
you know, tell me congratulations.

874
00:54:10,280 --> 00:54:14,400
But I don't feel like
<font color="

875
00:54:14,400 --> 00:54:17,600
Like... You're not walking away
from this a winner.

876
00:54:17,600 --> 00:54:23,520
I would say it's more just...
justice was served. Yeah.

877
00:54:26,160 --> 00:54:28,560
Tesla plan to appeal the ruling.

878
00:54:43,760 --> 00:54:46,480
Meanwhile, in the UK,

879
00:54:46,480 --> 00:54:50,080
<font color="
their self-driving cars

880
00:54:50,080 --> 00:54:51,880
in London later this year.

881
00:54:54,480 --> 00:54:57,200
This feels like so much more
high stakes than it did

882
00:54:57,200 --> 00:54:58,600
when I was in Phoenix.

883
00:55:00,240 --> 00:55:02,240
I mean, this is slightly
wild for me, right?

884
00:55:02,240 --> 00:55:04,680
<font color="
for 20 years,

885
00:55:04,680 --> 00:55:07,880
and we're driving in a driverless
car down Camden High Street.

886
00:55:07,880 --> 00:55:10,000
THEY LAUGH

887
00:55:07,880 --> 00:55:10,000
Like, this is...

888
00:55:10,000 --> 00:55:12,720
This is honestly something I thought
was much further in the future.

889
00:55:12,720 --> 00:55:14,200
It never gets old.

890
00:55:14,200 --> 00:55:18,760
<font color="
that the AI in driverless cars

891
00:55:18,760 --> 00:55:22,280
has come on leaps and bounds
in the past few years.

892
00:55:22,280 --> 00:55:24,800
And this is interesting.
This person's body language was

893
00:55:24,800 --> 00:55:27,200
sort of turning backwards
and forwards away from the crossing.

894
00:55:27,200 --> 00:55:28,680
<font color="

895
00:55:28,680 --> 00:55:31,280
The car sort of changed
its mind a couple of times.

896
00:55:31,280 --> 00:55:33,320
It saw the person turn their
body back and forth.

897
00:55:33,320 --> 00:55:36,560
The AI has got quite good
at learning that kind of intent.

898
00:55:36,560 --> 00:55:39,880
So, it's like super,
<font color="

899
00:55:39,880 --> 00:55:43,200
It's a completely different
experience from cruise control.

900
00:55:43,200 --> 00:55:45,240
Have I just undercut
your product there?

901
00:55:45,240 --> 00:55:49,240
You're comparing a floppy disk
with a quantum computer.

902
00:55:51,920 --> 00:55:54,840
Think of AI as
<font color="

903
00:55:54,840 --> 00:55:58,960
When you think about the wheel,
the calculator, the computer -

904
00:55:58,960 --> 00:56:00,720
different tools that, as humanity,

905
00:56:00,720 --> 00:56:03,560
we've invented to push
forward society... Mm-hm.

906
00:56:03,560 --> 00:56:07,000
..I think intelligent machines
<font color="

907
00:56:13,480 --> 00:56:16,760
Whether you like it or not,
driverless cars are coming.

908
00:56:16,760 --> 00:56:19,200
I mean, they're here already.

909
00:56:19,200 --> 00:56:21,800
And I still think there's lots
of good to come from that.

910
00:56:21,800 --> 00:56:26,640
But to get to this point, we had
to go through that difficult phase.

911
00:56:26,640 --> 00:56:31,640
<font color="
which, I mean, by definition,

912
00:56:31,640 --> 00:56:34,640
learning involves making mistakes.

913
00:56:34,640 --> 00:56:37,880
It's just that these mistakes,
you know, these deaths,

914
00:56:37,880 --> 00:56:39,400
these lives that were ruined,

915
00:56:39,400 --> 00:56:42,160
I don't think that
they were inevitable.

916
00:56:42,160 --> 00:56:46,640
<font color="
an acceptable price to pay

917
00:56:46,640 --> 00:56:49,120
for a new technology.

918
00:56:49,120 --> 00:56:51,520
I think different choices
could have been made.

919
00:56:51,520 --> 00:56:55,080
I think there were different ways
to balance the roll-out of

920
00:56:55,080 --> 00:56:58,440
the new product with
<font color="

921
00:56:58,440 --> 00:57:02,480
And I just hope now that
these are lessons from the past,

922
00:57:02,480 --> 00:57:06,400
you know, I hope that the steepest
part of this learning curve

923
00:57:06,400 --> 00:57:08,080
is now behind us.

924
00:57:53,280 --> 00:57:55,440
SIRENS WAIL

925
00:57:55,440 --> 00:57:58,280
The chief executive of one
<font color="

926
00:57:58,280 --> 00:58:01,920
health insurance companies has been
shot and killed in New York.

927
00:58:01,920 --> 00:58:05,680
The alleged killer has been
identified as Luigi Mangione.

928
00:58:05,680 --> 00:58:09,840
The suspect had something to say
about the insurance industry.

929
00:58:09,840 --> 00:58:13,440
The insurance companies are relying
<font color="

930
00:58:13,440 --> 00:58:15,200
to deny patient care.

931
00:58:15,200 --> 00:58:16,960
Stop denials with AI!

932
00:58:16,960 --> 00:58:19,600
ALL: How many people have to die?

933
00:58:19,600 --> 00:58:22,440
The anger and the upset is real.

934
00:58:22,440 --> 00:58:24,400
ALL: Health care is a human right!

935
00:58:26,240 --> 00:58:32,040
To discover more about AI and how it
<font color="

936
00:58:36,360 --> 00:58:39,120
..or scan the QR code on
the screen now.


